Papers with Self-Instruct framework
KNN-Instruct: Automatic Instruction Construction with K Nearest Neighbor Deduction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating synthetic instructions for large language models suffer from stale distribution and scalability. |
| Approach: | They propose a method which incorporates KNN deduction to produce meaningful new instructions by summarizing and learning from existing ones. |
| Outcome: | The proposed method outperforms all 7B models on the LMSYS leaderboard. |
What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate long-context instruction-tuning data are limited by poor quality and fewer than 35% of samples are multi-hop . |
| Approach: | They propose a framework that integrates a quality verification agent, a single-hop question generation agent, and a multi-hop questions merger agent to enhance model performance. |
| Outcome: | The proposed framework significantly improves data quality with high-quality, multi-hop, and diverse data. |